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Abstract 

This study examined how intellectual openness and gender affected the extent to 
which students engaged in dialectic critical discourse in computer-supported 
collaborative argumentation (CSCA). This study found: a) indications of 
differences in the number of personal rebuttals posted in reply to direct challenges 
between more versus less open students (ES = +0.33); b) significant differences in 
the number of rebuttals posted between males and females (ES = +0.66 ); c) 
significant differences in number of personal rebuttals posted between the less 
open versus more open students among the males (ES = +1.32), but not among the 
females (ES = -0.19); and d) no difference in the types of responses posted in reply 
to challenges between more open versus less open students. These findings 
illustrate how the effects of intellectual openness can be mediated by gender, or 
vice versa, how openness can potentially affect student performance in CSCA, 
and how process-oriented strategies can be used to analyze and structure 
discussions to promote critical discourse. 

Resume 


Cette etude examine comment l'ouverture d'esprit et le genre affecte l'engagement 
des etudiantes et etudiants dans un discours dialectique critique d'une 
argumentation collaborative soutenue par ordinateur (ACSO). II a ete trouve : a) 
des indications de differences dans le nombre de refutations personnelles placees 
en reponse a des contestations entre des etudiants plus ouverts par rapport a des 
etudiants moins ouverts d'esprit (ES = +0,33), b) des differences significatives 
entre le nombre de refutations placees par les hommes et les femmes (ES = +0,66), 
c) des differences significatives dans le nombre de refutations personnelles 
placees par les moins ouverts d'esprit et les plus ouverts d'esprit parmi les 
hommes (ES = +1.32), mais pas parmi les femmes (ES = -0,19), et d) pas de 
difference dans les types de reponses placees en reponse aux contestations entre 
les etudiants plus ouverts d'esprit et les moins ouverts d'esprit. Ces resultats 
illustrent comment les effets de l'ouverture d'esprit peuvent etre influences par le 
genre, ou vice versa, comment l'ouverture d'esprit peut potentiellement affecter la 
performance etudiante en ACSO, et comment les strategies orientees sur le 
processus peuvent etre utilisees pour analyser et structurer les discussions pour 
promouvoir le discours critique. 
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Introduction 

Collaborative argumentation is an instructional activity used to foster 
critical reflection and critical discussion (Johnson & Johnson, 1992) in both 
face-to-face (F2F) and online environments. Argumentation involves the 
process of building arguments to support a position, considering and 
weighing evidence and counter-evidence, and testing out uncertainties to 
extract meaning, achieve understanding, and examine complex ill- 
structured problems (Cho & Jonassen, 2002). This process not only plays 
a key role in increasing students' understanding but also in improving 
group decision-making (Lemus, Seibold, Flanagin & Metzger, 2004). At 
this time, online discussion boards are being increasingly used to engage 
learners in dialogue in both online and face-to-face courses in order to 
promote more in-depth discussions (Tallent-Runnels et al., 2006). 
However, studies show that the quality of online discussions is often 
shallow (Pena-Shaff, Martin, & Gay, 2001). As a result, a growing number 
of researchers are examining ways to promote critical thinking in online 
discussions by using computer-supported collaborative argumentation 
(CSCA) — a set of online environments and procedures designed to 
scaffold and guide students through the processes of argumentation. 

In CSCA, constraints are imposed on the types of messages students 
can post to a discussion in order to guide other students through the 
processes of collaborative argumentation. For example, Jeong (2005a, 
2005b) presented students with a fixed set of message categories 
(arguments, challenges, supporting evidence, explanations) designed to 
foster argumentation and debate in an asynchronous threaded discussion 
board. Students were required to classify and label each message by 
inserting a tag corresponding to a given message category in the headings 
of each message prior to posting each message. Similarly, Jonassen and 
Remidez (2005) developed a threaded discussion tool called 
ShadowPDforum where message constraints are built into the computer 
interface, thus requiring students to select and classify the function of 
each message before messages are posted to discussions. This approach 
has been implemented in a number of communication tools to facilitate 
collaboration and group communication. Some of these tools include 
Belvedere (Cho & Jonassen, 2002; Jonassen & Kwon, 2001), CSILE 
(Scardamalia & Bereiter, 1996), ACT (Duffy, Dueber, & Hawley, 1998; 
Sloffer, Dueber & Duffy, 1999), Hermes (Karacapilidis & Papadiasi, 2001), 
FLE3 (Leinonen, Virtanen, & Hakkarainen, 2002), AcademicTalk 
(McAlister, Ravenscroft & Scanlon, 2004), and NegotiationTooli (Beers, 
Boshuizen, & Kirschner, 2004). 

Despite these efforts to promote more critical discourse, the findings in 
CSCA research have been mixed - with no conclusive evidence to show 
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that CSCA improves student performance and learning (Baker & Lund, 
1997). For example, message constraints (and other variations of this 
procedure) have been found to elicit more replies that elaborate on 
previous ideas, and produce greater gains in individual acquisition of 
knowledge (Weinberger, Ertl, Fischer, & Mandl, 2005). In another study, 
message constraints generated more supported claims and promoted 
greater knowledge of the argumentation process (Stegmann, Weinberger, 
Fischer, & Mandl, 2004). However, no differences were found in 
individual knowledge acquisition, students' ability to apply relevant 
information and specific domain content to arguments, and ability to 
converge towards a shared consensus. Furthermore, message constraints 
were found to inhibit collaborative argumentation — producing fewer 
challenges per argument than argumentation without message 
constraints (Jeong & Juong, 2007). 

Given that collaborative argumentation is both an intellectual and 
social activity, one possible explanation for the mixed findings is that 
students' personalities or dispositions to engage in argumentation have 
not been taken into account in previous studies. Studies show that 
students are reluctant to criticize the ideas of other students (Lampert, 
Rittenhouse, & Crumbaugh, 1996; Nussbaum, 2002). As a result, 
Nussbaum et al. (2004) examined the combined effects of personality 
traits and the use of prompts (e.g., "My argument is...", "On the opposite 
side...", "Explain why...") for supporting critical discussions online and 
found that when prompts were used, disagreements were expressed more 
often by students who were less open to ideas, less anxious, and less 
assertive than students who were more open to ideas, more anxious, and 
more assertive. Every unit increase in a group's average score on 
assertiveness, openness to ideas, and anxiety were found to reduce the 
odds of a disagreement by 13%, 13%, and 16%, respectively. Chen & 
Caropreso (2004) found that groups with high profile and mixed profiles 
(high and low) across the "Big Five" personality traits (extraversion, 
neuroticism, agreeableness, conscientiousness, openness) produced more 
two-way messages (messages that solicit and invite others to reply) than 
low and neutral profile groups. 

In addition, studies have found gender differences in dispositions to 
engage in argumentation and gender differences in performance in 
CSCA. Jeong & Davidson-Shivers (2006) found that females posted fewer 
rebuttals to the disagreements and challenges of females than males, and 
males posted more rebuttals to the challenges of females than females. 
This finding was consistent with findings from previous studies that 
show that men tend to assert opinions strongly as facts, place more value 
on presenting information using an expository style, are more likely to 
use crude language, violate online rules of conduct, and engage in more 
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adversarial exchanges (Blum, 1999; Fahy, 2003; Herring, 1999; Savicki, 
Lingenfelter, & Kelley, 1996). In contrast, females tend to hedge, qualify 
and justify their assertions (using words like 'maybe', 'possibly', and 
'perhaps'), express support of others, make apologies, and in general, 
manifest a more consensus-making orientation and epistolary style. 
Furthermore, females have been found to be slightly more agreeable and 
males to be slightly more assertive and open to ideas (Costa, Terracciano 
& McCrae, 2001). 

All of the behavioral characteristics associated with gender and the 
personality traits described above are likely to be inter-related to one 
extent or another. However, intellectual openness appears to be the one 
trait that is most closely associated with the intellectual (rather than the 
social-emotional aspects) and primary function of collaborative 
argumentation. Given that intellectual openness measures the extent to 
which a student is open to new ideas, needs intellectual stimulation, 
carries conversations to higher levels, looks for deeper meaning in things, 
is open to change, and is interested in many things (International 
Personality Item Pool, 2001), the purpose of this study was to examine the 
combined effects of intellectual openness and gender in CSCA by 
determining to what extent they affect how and how often students 
directly respond back to challenges and disagreements — particularly 
with responses that help to generate deeper and more critical discussions 
(e.g., aigument-*:hallenge-»no reply vs. counter-challenge vs. 
explain/justify). Given that no previous studies have compared the 
performance of students across different traits in terms of the number of 
times students post follow-up responses (or personal rebuttals) to direct 
challenges and the types of behaviors that students exhibit when faced 
with a direct challenge (e.g., ignore challenge and don't reply, or let others 
reply to challenge; counter-challenge; explain/ justify; support with 
evidence), this study used what might best be described as a process- 
oriented approach to determine the effects of intellectual openness and to 
what extent those effects are mediated by gender. 

Theoretical Framework and Assumptions 

The theoretical basis for examining message-challenge-rebuttal sequences 
was based on three assumptions of the dialogic theory of language 
(Bakhtin, 1981; Koschmann, 1996): 

a) meaning is re-negotiated and re-constructed as a direct result of 
conflict (or cognitive conflict) produced in social interactions; 

b) conflict is produced not by examining an utterance by itself, but by 
examining the relationship between utterances; and 

c) conflict (or specific message-response pairs that produce conflict) is 
the primary force that triggers and drives critical discourse. 
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Support for this theory can be drawn from extensive research on 
collaborative learning that shows conflict and the consideration of 
both sides of an issue is needed to drive inquiry, reflection, 
articulation of individual viewpoints and underlying assumptions, 
and to achieve deeper understanding (Johnson & Johnson, 1992; 
Wiley & Voss, 1999). The need to explain, justify, or understand is 
felt and acted upon only when conflicts or errors are brought to 
attention (Baker, 1999). 

As a result, the assumptions in this study are: 

a) conflict is produced not by ideas presented in one message alone, 
but by the juxtaposition of opposing messages 

(e.g., argument-* challenge, challenge-* counter challenge); and 

b) conflicts produced in these exchanges help trigger subsequent 
responses that serve to dismiss or rebuke challenges 

(e.g., argument-* challenge-»counterchallenge), or verify 
(e.g., argument-*challenge-»evidence) and justify 
(e.g., argument-*challenge-*explain) arguments. 

These assumptions determined the approach used in this study to 
examine how intellectual openness and gender affect the frequency and 
the types of responses posted in reply to challenges — particularly the 
types and sequences of responses that help to produce critical discourse 
and construct deeper meaning. 

Research Questions 

This study examined the combined effects of intellectual openness and 
gender on group performance in CSC A by addressing two questions: 

1) How do these two traits affect the number of times each student posts 
personal rebuttals in response to direct challenges? 

2) How do these two traits affect the way students respond to direct 
challenges (with counter-challenges, explanations, supporting 
evidence, no response)? 


Method 


Participants 

The participants were graduate students (n = 54) from a major university 
in the Southeast region of the U.S., consisting of 35 females and 19 males, 
and ranging from 20 to 50 years in age. The participants in this study were 
enrolled in a 16-week online graduate introductory course on distance 
education during the fall 2004 term (8 females and 6 males), spring 2005 
term (9 females and 5 males), fall 2005 term (11 females, 5 males), and 
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spring 2006 term (7 females, 3 males). The four cohort groups were 
examined collectively to obtain a sufficient corpus of data for this study. 
Given that the majority of the students took the course at a distance, few 
if any of the students had previous opportunities to interact with one 
another outside of the online environment. The performance of 10 
students was omitted from analysis because four students did not 
complete the instrument used to measure intellectual openness and six 
students dropped the course and did not participate in all the group 
discussions examined in this study. 

Debate Procedures 

The students participated in four weekly online team debates using 
asynchronous threaded discussion forums in Blackboard™, a web-based 
course management system. The debates were structured so that: a) 
student participation in the debates and other discussions throughout the 
course contributed to 20% of the course grade; b) for each debate, 
students were required to post a minimum of four messages; c) prior to 
each debate, students were randomly assigned to one of two teams 
(balanced by gender) to either support or oppose a given position; and d) 
students were required to vote on the team that presented the strongest 
arguments following each debate. The purpose of each debate was to 
critically examine design issues, concepts and principles in distance 
learning covered during the week of the debate. The students debated 
claims such as: "The Dick & Carey ISD model is an effective model for 
designing the instructional materials for this course", "Type of media 
does not make any significant contribution to student learning", "Given 
the data and needs assessment, the fictitious country of NED should not 
develop a distance learning system", and "Print is the preferred medium 
for delivering a course study guide". 

Online Debate Messages and Message Labels 

Students were presented a list of four message categories (see Figure 1) 
during the debates to encourage students to support and refute presented 
arguments with supporting evidence, explanations, and challenges 
(Jeong & Juong, 2007). Based loosely on Toulmin's (1958) model of 
argumentation, the response categories and their definitions were 
presented to students prior to each debate. Each student was required to 
classify each posted message by category by inserting the corresponding 
label into the subject headings of each message (along with a short 
descriptive title representing the main idea presented in the message), 
and restrict the content of each message to address one and only one 
category or function at a time. The investigator occasionally checked the 
message labels to determine if students were appropriately labeling their 
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messages according to the described procedures. Students were 
instructed to return to a message to correct errors in their labels. No 
participation points were awarded for a given debate if a student failed to 
follow these procedures. 


Label Description of Label 

+ Identifies a message 

posted by a student assigned to the 
team supporting the given 
claim/statement 

Identifies a message 
posted by a student assigned to the 
team opposing the given 
claim/statement 

ARG# “Identifies a message that 
presents one and only one argument 
or reason for using or not using chats 
instead of threaded discussion 
forums). Number each posted 
argument by counting the number of 
arguments already presented by your 
team. Sub-arguments need not be 
numbered. ARG = “argument” 

EXPL Identifies a reply/message that 
provides additional support, 
explanation, clarification, elaboration 
of an argument or challenge. 


BUT Identifies a reply/message that 
questions or challenges the merits, 
logic, relevancy, validity, accuracy or 
plausibility of a presented argument 
(ARG) or challenge (BUT) 

EVID Identifies a reply/message that 
provides proof or evidence to establish 
the validity of an argument or 
challenge. 


Example Message by Label 


-ARG1 One’s choice of media makes 
very little difference in students’ 
learning because the primary factor 
that determines level of learning is 
one’s choice of instructional method. 


-EXPL As a result, media are merely 
vehicles that deliver instruction but do 
not influence student achievement any 
more than the truck that delivers our 
groceries causes changes in our 
nutrition.” 

+BUT However, one’s choice of media 
can affect or determine which 
instructional methods are or are not 
used. If that is the case, then choice of 
media can make a significant 
difference. 

-EVID Media studies, regardless of the 
media employed, tend to result in “”no 
significant difference”conclusions 
(Mielke, 1968).” 


Figure 1. Example instructions on labeling messages in the online debates. 
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SUPPORT statement because... 

Instructor 

Sat Oct 2, 2004 11:18 am 

. +ARG1 MedialsButAMere... 

Student name 

Mon Oct 4, 2004 8:47 pm 

. . -EVID MedialsButAMere... 

Student name 

Tue Oct 5, 2004 7:09 pm 

. . . +BUT Relativity Theory... 

Student name 

Tue Oct 5, 2004 9:43 pm 

. . . . -BUT Relativity... 

Student name 

Sat Oct 9, 2004 10:12 am 

. . -BUT Whataboutemotions? 

Student name 

Tue Oct 5, 2004 9:53 pm 

. . +EVID DistEdEffectiveAsF2F 

Student name 

Tue Oct 5, 2004 10:40 pm 

. . -BUT Mediaamerevehicle 

Student name 

Wed Oct 6, 2004 8:19 pm 

. . +EVID MooreConcurs 

Student name 

Wed Oct 6, 2004 10:07 pm 

. . . +EXPL MediaSelection... 

Student name 

Sun Oct 10, 2004 12:35 am 

. . . -BUT WellChosen... 

Student name 

Sun Oct 10, 2004 4:31 pm 

.... +BUT Supporting... 

Student name 

Sun Oct 10, 2004 5:37 pm 

. . -BUT Mediaismorethe... 

Student name 

Fri Oct 8, 2004 5:30 pm 

. . . +BUT Supporting... 

Student name 

Sat Oct 9, 2004 8:51 am 

. . -BUT LearningNotSimply... 

Student name 

Mon Oct 11, 2004 9:54 am 

. +ARG2 Standards for. . . 

Student name 

Wed Oct 6, 2004 1 :48 pm 

. . +BUT Clarification? 

Student name 

Sun Oct 10, 2004 5:39 pm 

. +ARG3 MediaUnrelatedto... 

Student name 

Wed Oct 6, 2004 3:12 pm 

. . -BUT MediaUnrelatedto... 

Student name 

Wed Oct 6, 2004 8:26 pm 

. . . +BUT MediaSelection 

Student name 

Thu Oct 7, 2004 9:20 am 

. . . . -BUT MediaSelection 

Student name 

Sun Oct 10, 2004 11:21 am 

. . +EVID MethodNotMedia 

Student name 

Wed Oct 6, 2004 11:04 pm 

. . -BUT MediaUnrelatedto... 

Student name 

Sat Oct 9, 2004 10:59 am 

Figure 2. Example debate with labeled messages in a Blackboard™ 
threaded discussion forum. 


/Vote: The names of students have been removed to protect students' confidentiality. 
The discussion thread for posting arguments to oppose the given statement (“OPPOSE 
statement because ...”) is out of view in the above illustration. 


Students were also instructed to identify each message by team 
membership by adding for opposing or "+" for supporting team at the 
end of each label (e.g., +ARG, -ARG). These tags enabled students to 
easily locate the exchanges between the opposing and supporting teams 
during the debates (e.g., +ARG -* -BUT) and respond to the exchanges to 
advance their team's position. In Figure 2 is an example of how the 
labeled messages appeared in the discussion board. One discussion 
thread was designated for posting supporting arguments, and a second 
but separate thread (not shown in Figure 2) was designated for posting 
opposing arguments. Figure 3 provides an excerpt from one of the 
debates to illustrate some of the messages that were coded for each 
message category. 




INTELLECTUAL OPENNESS 


9 


Personality Trait Instruments 

Students were measured on intellectual openness using a ten-item 
instrument (see Table 1) from the International Personality Item Pool 
(2001) with a reliability coefficient of .82. To measure the level of 
intellectual openness, students were asked to rate to what extent they 
agreed (on a scale of one to five) with statements like "open to new ideas", 
"need intellectual stimulation", "carry the conversation to a higher level", 
"look for a deeper meaning in things", "am open to change", and "am 
interested in many things". The scores across all ten items were added to 
compute a total score for each student. The median score among all 
students in the study was used to determine which students were 
classified as low and high on intellectual openness. The mean score for 
intellectual openness was 9.3 (SD = 5.41, n = 54) with a minimum score of 
-6 and maximum score of 20. The differences in scores between the male 
(M = 10.68, SD = 4.69, n = 19) and female students (M = 8.54, SD = 5.69, 
n = 35) were not statistically significant, t(52) = 1.40, p = .167. 


Table 1 

Ten-item scale used to measure intellectual openness 


Am interested in many things 

+1 

Am open to change 

+1 

Carry the conversation to a higher level 

+1 

Prefer variety to routine 

+1 

Want to increase my knowledge 

+1 

Am not interested in abstract ideas 

-1 

Am not interested in theoretical discussions 

-1 

Prefer to stick with things that 1 know 

-1 

Rarely look for a deeper meaning in things 

-1 

Try to avoid complex people 

-1 


The Data Set 

ForumManager (Jeong, 2005c) was used to download the student-labeled 
messages from the Blackboard™ discussion forums into Microsoft Excel 
while maintaining the hierarchical threaded format in order to preserve 
the information needed to determine which responses were posted in 
reply to which messages. The Discussion Analysis Tool or DAT (Jeong, 
2005b & 2005d), was then used to automatically extract the codes 
assigned to each message from the subject headings to tag each message 
an argument (ARG), evidence (EVID), challenge (BUT), or explanation 
(EXPL). The data analyzed in this study (see Figure 4) consisted of 1247 
messages (not including the 51 instructor's messages / prompts and 14 
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Category Message Text 


ARG Borje Holmberg's Theory of Interaction and Communication states 

that "learning pleasure supports student motivation" and and "strong student 
motivation facilitates learning"(Simonson, p. 43). I would argue that compelling 
media and multi-media increases learning pleasure and thus facilitates student 
learning - Bob 

+BUT Bob, what research is available to support your statement 

"compelling media and multi-media increases learning pleasure and thus 
facilitates student learning"? 

+EVID "Extensive research findings indicate that no direct link has been 

established between delivery medium, level of interaction, and the effect of 
both on student achievement." Keast 1997."...Kozma (1994) agrees with me 
that there is no compelling evidence in the past 70 years of published and 
unpublished research that media cause learning increases under any 
conditions. Like all other researchers who have made a careful study of the 
arguments and research studies (e.g., Winn, 1990), he reaches a conclusion 
that is compatible with my claims (Clark, 1983)." 

BUT From my perspective, Clarke's "Media Will Never Influence 

Learning" does not take into account the effect poor media has on 
learning. I have attended many a training session where the media was 
deplorable to say the least. While the content was there, I did not learn 
very much (if anything) because I was fighting the quality of the media. I 
would argue that if poor media can have detrimental effect, then good 
media can have positive effect on learning - Bob 

-EXPL Please refer to a report by Harold F. O'Neil, Univ. of Southern 

California, for the Office of Naval Research entitled "What Works in Distance 
Learning" Feb 23, 2003. The report offers a guideline (p. 37) for a 
multimedia strategy. I quote "People learn better from corresponding words 
and graphics (e.g., animation, video, illustrations, pictures) than from words 
alone". This report guideline is based on research conducted by R. E. Mayer 
and R. B. Anderson and published in the Journal of Educational Psychology 
83, 484-490 and 84, 444-452. 1 would argue that more recent research is 
showing that multimedia contributes to learning. Thanks, Bob. 

-EVID Bob's -ARG5 talks to the research of Hilary Perraton in that 

multimedia provide more "effective" learning experiences. The pleasurability of 
the experience does support the effectiveness of the learning. 


Figure 3. Example of a coded thread generated by an argument posted in opposition to 
the claim "Media makes very little or no significant contributions to learning" 

ARG = argument , BUT = challenge , EVID = supporting evidence, EXPL = explanation 
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Figure 4. Interaction data produced by DAT software. 





Example: 48% of replies to opposing arguments (-ARG) were challenges (+BUT). 


messages that were not assigned a label by students). DAT was then used 
to tally the number of specific types of responses elicited by each specific 
type of message (e.g., number of challenges posted in response to each 
observed argument) to generate the raw scores needed to test the effects 
of intellectual openness. 

To determine to the extent students were able to correctly assign labels 
to each message posted to the debates, one debate from each course was 
randomly selected and coded by the investigator to test for errors in the 
labels. Overall percent agreement was .91 based on the codes of 158 
messages, consisting of 42 arguments, 17 supporting evidence, 81 
challenges, and 17 explanations. The Cohen Kappa coefficient, which 
accounts for chance in coding errors based on the number of categories in 
the coding scheme, was .86 — indicating excellent inter-rater reliability 
(Bakeman & Gottman, 1997, p. 66). 

Observed in the discussions were a total of 657 instances where 
messages elicited one or more challenges. In 514 (78%) of these 657 
instances, the individual student receiving the direct challenge did not 
post a personal rebuttal to the challenge. In the remaining 143 (22%) of the 
657 instances, a personal rebuttal was posted in reply to the challenge. Of 
these 143 instances where students posted follow-up responses to 
challenges, 112 were counter-challenges, 29 were explanations, and 4 


12 


INTELLECTUAL OPENNESS 


were supporting evidence. No outliers were found (beyond 3 standard 
deviations) in the number of personal rebuttals posted by the 54 students 
observed in this study. 


Results 


Frequency of Personal Rebuttals 

The results of a 2 (low vs. high openness) x 2 (male vs. female) univariate 
analysis of variance revealed no significant difference in the number of 
personal rebuttals between more open versus less intellectually open 
students, F(l, 50) = 3.55, p = .065. However, the observed difference did 
approach statistical significance. The more open students posted 45% 
more personal rebuttals (M = 2.62, SD = 2.61, n = 29) than the less open 
students (M = 1.80, SD = 2.40, n = 25), with effect size of +0.33. At the same 
time, the results of post-hoc tests showed that the more open students left 
33% more challenges without rebuttals (M = 10.76, SD = 5.59, n = 29) than 
the less open students (M = 8.08, SD = 5.61, n = 25), t( 52) = -1.75, p = .086, 
effect size of -0.48. 

A significant difference was found in the number of personal rebuttals 
posted by male versus female students, F(l, 50) = 4.66, p = .036. Males 
posted nearly twice the number of personal rebuttals (M = 3.31, SD = 2.73, 
n = 19) than females (M = 1.66, SD = 2.23, n = 35), with effect size of +0.66. 

The interaction between openness and gender was also found to be 
significant, F(l, 50) = 6.50, p = .014. This finding shows that the effects of 
intellectual openness were substantially influenced by gender. Among the 
males, students who were more intellectually open posted 1.8 times more 
personal rebuttals (M = 4.54, SD = 2.94, n = 11) than those who were less 
intellectually open (M = 1.62, SD = 1.06, n = 8), with effect size of +1.32. 
This finding, however, must be interpreted with caution given the small 
number of males observed in this study. Among the females, students who 
were more intellectually open posted about 23% fewer (not more) personal 
rebuttals (M = 1.44, SD = 1.50, n = 18) than students who were less 
intellectually open (M = 1.88, SD = 2.85, n = 17), with an effect size of -0.19. 

Types of Responses to Challenges 

The number of times no personal rebuttals were posted in reply to a 
challenge and the number of times challenges, explanations, and 
responses with supporting evidence was posted in reply to challenges 
was counted for each individual student. To control for variance in the 
observed counts across individual students, the actual counts for each 
response type for each individual were translated into relative 
frequencies for each individual (e.g., 3, 5, 1, 1 was translated to .30, .50, .10, 
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10). The relative frequencies were then multiplied by a value of 10. The 
adjusted frequencies were then totaled for each response category across 
all students who were less intellectually open, and then totaled again for 
all students who were more intellectually open to produce the following 
numbers: 25 BUT, 8 EXPL, 1 EVID, and 197 No reply (or .10 BUT, .03 
EXPL, .01 EVID, .86 No reply) produced by the less open students, and 49 
BUT, 8 EXPL, 2 EVID, and 231 No reply (or .16 BUT, .03 EXPL, .01 EVID, 
and .80 No reply) produced by the more open students. Using the 
Chi-Square test of independence, this study found no significant 
differences in the way students responded to direct challenges between 
students who were more open versus less intellectually open, /} (3) = 4.74, 
p = .19. 

Post-hoc tests used to determine the effects of gender on response 
patterns could not be conducted due to an insufficient number of male 
participants (n = 6) that were both low in intellectual openness and 
received at least one opportunity to reply to a direct challenge with a 
personal rebuttal. However, Figure 5 reveals that within the exchanges 
between the more open students only, the students were more likely to 
respond to arguments with challenges and respond to challenges with 
counter-challenges than in the exchanges between less open students 
only. This finding (although speculative in nature) suggests potential 




Between less open students 


Between more open students 


Figure 5. Transitional state diagrams of response patterns in exchanges 
produced by students within group. 
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differences in performance between groups that consist only of students 
who are more open and students who are less open. 

Discussion 

The purpose of this study was to examine how intellectual openness and 
gender affected the way students responded to direct challenges and to 
what extent their responses exhibited critical discourse in CSCA. This 
study found no difference in the number of personal rebuttals posted in 
response to direct challenges between the more versus less intellectually 
open students. However, this study found that the males posted nearly 
twice the number of personal rebuttals than the females. This study also 
found that the effect of intellectual openness depended on the gender of 
the student posting rebuttals to direct challenges. The more open male 
students posted nearly twice the number of personal rebuttals than the 
less open male students — a difference that was found to be large and 
substantial. In contrast, the more open female students were found to 
have posted fewer personal rebuttals than the less open female students. 
Finally, no differences were found in the way students responded to direct 
challenges between students with low versus high intellectual openness. 
More intellectually open students, for example, were just as likely to react 
to a direct challenge by posting a counter-challenge, explanation, 
supporting evidence, or not posting a follow-up response, as the less 
intellectually open students. 

Given that no significant differences were found in the number of 
rebuttals posted by more versus less intellectual open students, this 
finding was not consistent with the findings of Nussbaum et al (2004) 
who had found that disagreements were expressed more often by 
students who were less intellectually open. However, this study did find 
significant differences between genders — a finding that is consistent 
with the findings reported in a previous study (Jeong & Davidson- 
Shivers, 2006) that showed male students posted more rebuttals in reply 
to challenges than female students. But more importantly, this study 
found that gender played a significant role in mediating the effects of 
intellectual openness. Level of intellectual openness was found to have a 
large and substantial effect on the number of times students posted 
personal rebuttals, but only for the male students and not for the female 
students. As a result, these findings suggest that measuring intellectual 
openness may be useful only when implemented with male participants, 
but not with female participants. 

One possible explanation as to why intellectual openness influenced 
the performance of males but not females may lie in the specific items 
used in the scale to measure openness. For example, items like "open to 
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new ideas" and "open to change" could theoretically serve more as a 
measure of students' agreeableness and willingness to accept opposing 
viewpoints without question and further discussion - a trait that is 
observed more often among females, not males (Costa, Terracciano & 
McCrae, 2001; Jeong & Davidson-Shivers, 2006). In contrast, the trait 
"carry conversations to a higher level" seems to be a sub-trait that has a 
more direct association with how and how often students responded to 
challenges. At the same time, there is the possibility that some of the sub- 
traits in the scale may be more dominant (e.g., "carry conversations to a 
higher level") or less dominant (e.g., "open to new ideas" and "open to 
change") among males, and vice versa for female students. As a result, the 
ten-item scale used in this study may need to be refined (or even 
simplified) before it can be used to accurately gauge and predict the 
performance of both males and females participants in CSCA. 

Nevertheless, the findings in this study provide some answers and 
guidelines as to why, when, and how to use measures of intellectual 
openness (or other personality traits) and gender to predict how students 
will perform in CSCA and to identify which students need direct 
interventions to promote more critical discourse. Specifically, the findings 
suggest that intellectual openness can serve as a useful criterion for 
assigning male students to mixed-gender or all-male groups in order to 
maximize exchanges between students that result in deeper inquiry and 
the construction of deeper meaning and understanding. Although the 10- 
item scale used to measure intellectual openness is very short and not 
necessarily difficult to implement, what is still needed at this time is an 
online tool that can automate the task of surveying students for one or 
more instructor-selected traits, assigning students across a given number 
of groups using random stratification to balance groups across selected 
traits, and notifying students of their group assignments. Ultimately, the 
findings and process-oriented methods described in this study can 
potentially be used to formulate pedagogical rules and stochastic models 
that can be implemented in future CSCA systems — systems that might 
one day provide the tools to automate the tasks of assigning students to 
groups based on observed traits, diagnosing student performance, and 
delivering interventions to optimize group performance. 

However, the findings in this study are not conclusive due to a 
number of limitations in the scope and design of the study. Future studies 
will need to examine: a) a larger sample to compare performance between 
male and female students who score in the lower versus upper quartiles; 
b) the effects across different task structures other than those used in this 
study to encourage argumentation (e.g., message constraints, assigning 
students to opposing teams, requiring minimum number of postings); c), 
smaller discussion groups where larger deviations in trait scores are more 
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likely to affect group performance; d) the effects across a wider range of 
exchanges (e.g., challenge^ concede, challenge^ derogatory remark) that 
mark both constructive and non-constructive interactions; e) differences 
in performance produced by other traits more closely associated with the 
intellectual dimensions of argumentation such as active versus reflective 
learning styles (work in progress); and f) how the traits of both the 
messenger and challenger affect the frequency of rebuttals posted by the 
messenger. 

Overall, this study provides insights into how differences in 
personality traits and gender can affect group performance in CSCA. 
Specifically, this study was successful in what was an initial attempt to 
determine if, when, and how intellectual openness and gender together 
affect the specific processes and message-response exchanges that define 
and promote critical thinking in CSCA. The methods and software tools 
used in this study to measure the combined effects of intellectual 
openness and gender present a unique and potentially useful approach to 
developing and empirically testing instructional interventions that use 
information on learner characteristics to predict, diagnose, intervene, and 
optimize group performance in computer-supported collaborative 
learning, work, decision-making, and problem-solving. 
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